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ABSTRACT 


Any prior information an experimenter has should be incorporated into the 
design of further experiments. Assume such information is expressed as 
probabilities that each parameter of the full factorial model is nonzero. 
This report develops optimal design procedures by posing the experimental 
design problem as a finite decision problem. Bayes and mini -max design 
strategies are then derived and their application illustrated. The major 
computational step is the evaluation of all possible matchings of physical 
variables to the abstract variables of all potential designs. The technique 
of telescoping sequences of blocks permits the consideration of experiments 
to be performed in stages. 


SIMBOLS 
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B(h) 
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S(i,h) 
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p b 

p i 


U 

u(h) 


complete set of parameters or coefficients 

subgroup of B used at the h^ stopping point of experiment 
denotes the stopping point of the experiment 

the set of standard order subscripts of the elements of p.(x)B(h) 

number of factors or independent variables 

number of treatment combinations at the h'*'* 1 stopping point 

prior probability of a block effect not being zero 

prior probability that p f 0 

probability the experiment terminates at exactly the h. 
stopping point 

total expected utility for a given strategy 

total expected utility of the h^ stopping point for a given 
strategy 
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a.@B(h) 


expected utility gained by assigning the estimator for 

p. @f{ a) to P k 

expected utility gained by assigning the estimator for p.(x)B(h) 
to the block effect that alias set is confounded with 

utility assigned to an unbiased estimate of p_^ at the h^ 
stopping point of the experiment 

independent variables (design) 

dummy independent variables which are identically equal to one 
for all treatment combinations 

independent variables (physical) i = 1,2,. ..,n 

a group operation 

random response or dependent variable 
observed value of Y ••• 

parameters of a model equation in the design variables 
parameters of a model equation in the physical variables 
constant terms of the model equation 
estimates of the parameters p^, . . . 
estimates of the parameters p^, . . . 

coset obtained by multiplying all elements of B(h) by fh 

random error 

INTRODUCTION 


In a two -level full factorial experiment with n independent variables 
X A ,X,_, ... there are 2 n possible distinct treatment combinations. If 
all the treatment combinations are performed, it is possible to estimate 
the P's in the model equation of the form 
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1 = Pi + ^A + % + P AB X A X B + P C X C 

+ ^AC X A X C + ^BC X B X G + ^ABC^aVc + ^ 1 ' 


where 6 is a random variable with mean zero and finite variance. 

It will be convenient at this point to introduce an alternate notation for 
equation (l). Let the n independent variables be denoted as X , . . . , X . 
Number the 2 n P's in equation (l) from P^ to P n and consider the 
following equation similar to equation (l). 2 -1 


Y = P 0 + P^ f P 2 X 2 + P 3 X lX2 + p^ + 


+ • • • + p X lXo ... X +5 

2 n_i 1 2 


(2) 


Equation (l) and equation (2) are both written in what is called the standard 
order. If the subscripts of the P's are rewritten as n-digit binary 
numbers j it becomes quite obvious how the terms and coefficients of equation 
(2) are related to their standard order number. For example., let n = 4 
and consider the following equation where the subscripts on the P's are 
written as binary numbers. 



W 3 + p 


+ P 10 X 2 


l0l X 3 X i 


+ p 


+ P 


•\r v 
11 X 2 X 1 

110 3 2 


+ ^lll X 3 X 2 X l + P iooo x 4 + P 1001 X 4 X 1 ^ 

+ p 1010 x 4 x 2 + P 101lWl + P 1100 X 4 X 3 


P 1101 X 4 X 3 X 1 + P 1110 X 4 X 3 X 2 


+ p llll X 4 X 3 X 2 X i 


In general a p whose subscript in binary notation has ones in the i , i , . . 
i fc locations from the right is the coefficient of the X. X. . .'.X. 
interaction. X 1 X 2 1 k 
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It Is well known that the set of all 2 n coefficients or parameters form 
a group denoted B under the appropriate operation denoted (x) . In the 
alphabetic notation this operation is simply commutative multiplication 
of the letters with the exponents reduced modulo 2. In the binary type 
notation this operation may also be denoted 0 and is defined as 




mm 

n n-1 




k k , . . .k_ 
n n-1 1 


d d. • d~ 

n n-1 1 


where d. = (k. + m. )(mod 2 ) 

i l i ' ' ' 


Thus for example 


p 03M© P DCB = P dg 2 b 2 a = ^DA 

and :■ 

f3 0111 © P U10 " ©.001' 


A regular fractional replicate of the full factorial design does not allow 
separate estimation of all of the P’s. Certain linear combinations of 
them can be estimated however. The aliased sets of parameters which can 
be estimated depends upon the treatment combinations composing the fractional 
replicate, or equivalently upon the choice of the defining parameter group 
(d.p.g. ) 


Holms and Sidik (ref. l) present a discussion of double (and multiple) 
telescoping sequences of blocks. Telescoping allows an experimenter to 
perform a factorial experiment in stages where the starting stage is a 
small regular fractional replicate and the final stage is some larger 
regular fraction. Each succeeding stage adds treatment combinations to 
those already performed in earlier stages. In order to retain orthogonality 
and orthogonal blocking each stage must be a power of two times the size 
of the preceding stage and all the treatments run must themselves form a 
regular fraction. This requirement implies that the d.p.g. defining the 
regular fraction at any stage must be a subgroup of the d.p.g. of the 
previous stage. Since multiple telescoping allows several possible choices 
of subgroups from each preceding group and each such choice may correspond 
to a potential stopping point we will simply index the stopping points by 
h and ignore the relations among the groups. Thus the d.p.g. 's that 
define fractional replicates at the stopping points are subgroups B(h) 
of B. The aliased sets of parameters at each stopping point are the 


IQ S 0 1 £ 


3(h) 
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At this point a distinction between physical and design variables will be 

made. The physical variables in an experiment will be denoted as X g , . . . , 

X . Each X. is chosen to represent one of the physical variables and is 
n i 

fixed for the remainder of the experiment. For example 
X- L = temperature 
X g = time 


X^ = velocity 

The design variables will be denoted as X^, X^, X^, . . . and so forth. 

These variables represent abstractions and tables exist (for example, the 
KBS tables (ref. 2) or Addelman’s tables (ref. 3 )) which tabulate experimental 
designs in terms of these design variables. When an experimenter consults 
one of these tables and chooses a design, he must then determine a matching 
of the design variables and the physical variables. Ordinarily the choice 
is arbitrary because the experimenter is not assumed to have any prior 
information available which would indicate one matching might be preferred 
to another. We consider the problem of choosing a best design under the 
following conditions of prior knowledge: 

1. For each the experimenter can specify p^ = P^P.^ f 0} • 

2. For each p. and each h denoting a possible stopping point of 
the experiment the experimenter can specify the value to him of obtaining 
an unbiased estimate of p^. This is denoted by u^(h) . 

3 . For each h denoting a possible stopping point of the telescoping 
experiment the experimenter can specify p g ^ = probability of stopping exactly 
at the n stopping point. 

Recall that none of the P’s may be separately estimated from a fractional 
factorial experiment unless some assumptions about certain others of the 
P's are introduced. Conditions one and two above provide assumptions 
that will enable the experimenter to assign the estimator for an alias 
set to a single parameter from the alias set and evaluate the consequences 
of this . 


It is also evident that changing the matching of design and physical 
variables will change the alias sets . For if the matching for n = k is 


X 1 = X A 


X, 


X. 


B 


X. 


X, 


( 4 ) 


X, - 
h D 
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then the alias set {P A , (3-g, (3^, j3 D ) is mapped into {P 0001 , P Q 010^ 

p ioiv p iooo) = Or h’ P ll' • But the matchin s 





X. 


X, 


X. 


X 


'D 


X, 


( 5 ) 


maps {p A , Pg, P D } into {P 0010 , P 0001 > P 0 ur P 010o) “ { '?2 ’ P 1 J P 

The steps of the design and performance of the experiment may now he 
represented as a finite discrete game between the experimenter and nature. 

The decision space E for the experimenter is composed of the choice of 
initial defining parameter group, choice of sequence(s) of subgroups that 
define the telescoping, choice of physical-design variable matching, arid the 
choice of parameter-estimator matching. The decision space N for nature 
consists of the choice of which of the P's will be nonzero and the choice 
of the stopping point of the experiment. 

A flow diagram is presented in fig. 1 which indicates the sequence of 
choices to be made by the experimenter and nature. 


It should be noted that the order of presentation shown in fig. 1 of the 
choices of nature and the experimenter are not necessarily in temporal 
order. The choice of d.p.g. and physical-design variable matching are 
interchangeable but must be made before the actual execution of the 
experiment. Nature's choice of the parameter values would normally be 
considered to be made prior to anything the experimenter does. However, it 
does no harm to conceive of nature being permitted to choose the parameter 
values after the experiment is performed. In fact, this concept is more 
convenient in describing the Bayes decision procedure for the experimenter. 

Once given the structure of a game or a decision problem and the utilities 
of the outcomes, the statistician must devise a method for choosing among 
the possible alternatives (i.e., choose a strategy) so that a desirable 
outcome is finally attained. 

There are two general principles by which strategies are ordered which are 
in current use. One is the Bayes principle and the other Is the mini -max 

P T 8 i n til p 

-i— j. r w* -i* . a 
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The Bayes Solution 

The Bayes procedure assumes that nature is an indifferent participant in 
the game. Thus, the choices that nature makes are independent of the 
choices available to the experimenter but instead are made according to a 
specified probability distribution. Any information the experimenter may 
have concerning the strategy (i.e., the specified probability distribution) 
nature will follow is useful information that should be incorporated into 
the decision procedure. The strategy for the experimenter which maximizes 
the utility, given the assumed strategy for nature, is the Bayes strategy. 

Before considering how best to match physical and design variables let us 
assume that some matching has been made. Then consider the problem of 
matching estimators and parameters at the h^ 1 stopping point. The d.p.g. 
is B(h) and the alias sets are all those distinct cosets of the form 

P. © B(h) =(p , p , p } ( 6 ) 

12 m 

If the parameter p^e P- © B(h) and the estimator for that alias set is 
assigned to p^., then, assuming independence, the prior probability that 
the estimator will be unbiased is 

' [“(1 - Pj ) 

je S(i,h) (7) 

J r k 

"bll 

Since u^h) is the utility of an unbiased estimate of p. at the h 
stopping point 

U(i,k) = u fc (h) "j j" (l - p^) (8) 

je S(i,h) 

j / k 

Is the expected utility of the decision to assign the estimator for the 
alias set p^ © B(h) to the parameter (3, . Thus the Bayes strategy is 
to assign the estimator to the parameter or the alias set which maximizes 
this expected utility. A special case requires separate discussion. 

Suppose that an alias set is confounded with some block effect. Assume 
that prior probabilities can be assigned to the event that each block 
effect will be nonzero and assume the utility of an unbiased estimate of 
’each block effect can be specified. Then this information can. be incorporated 
Into the decision procedure by computing the two expected utilities. 

U(i,k) = \( h )~| |"(l - Pj) (1 - Pj,) 

je S(i,h) 

j f k 


(9) 



Ki/o) = u | |~ (1 - Pj ) (10) 

je 3(i,h) 

here p. _= p {the block effect 4 o} 

a, = utility of an unbiased estimate of the block effect and 

o 

U(i,b) = the expected utility gained by assigning the estimator for 
(S. (x) B(h) to the block effect. 


Normally one would expect = 0 and hence Tj( i , b ) = 0 thus not 
necessitating the calculation in equation (10). 

In connection with block effects it is also important to note that depending 
upon how the block parameters are defined, the estimator for the d.p.g. 

may be confounded with blocks. 

tthice the alias sets are disjoint and the estimators for each alias set are 
independent, of each other, it follows that the choice of parameter from one 
alias set should not influence the choice from another alias set. Thus for 
the chosen matching of physical and design variables, the chosen defining 
contrasty and the chosen estimator-parameter matching, the expected utility 
at the h. stage may be denoted 

u(h) = 2 v(l ’ k ^ (11) 

where the summation is over all the distinct cosets at the h ' stage, and 
k is the choice of k within each coset which maximizes U(i,k). By 
coriaition3 on page 5 it is also assumed that the experimenter can specify 
the probabilities of stopping exactly at each of the stopping points. Thus, 


U = 


P 3h 0(h) 


( 12 ) 


h 


represents- the total expected utility over all the stages given the choice 
" ■ defining parameter groups and physical -design variable matching. 


O -i. 


The formal Bayes procedure is then to compute U for all possible distinct 
choices of d.p.g. * s and physical-design variable matchings and use any 

choice which gives the maximum U. 

Mini -Max Solution 


rive a mini-max strategy, the statistician assumes nature to be an 
hjj . ve player who will choose alternatives available which will minimise 
aximum utility the statistician may gain. Thus the experimenter is the 
icing player and nature is the minimizing player. 
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A strategy for nature involves two component choices: the choice of which 

parameters will he nonzero and the choice of the path and stopping point 
of the telescoping. One choice available to nature which the experimenter 
has no influence upon is the choice of nonzero parameters. In fact, it is 
possible that nature may choose to let every parameter be nonzero. In this 
instance it will not be possible to obtain unbiased estimates for any 
parameter until the full replicate is run. If the utility function is 
restricted to be zero when the estimator is biased, then there is zero 
utility except for the full replicate. Thus, the only possible way for the 
experimenter to obtain a gain is to design for the full replicate. However, 
nature also may choose the stopping stage and so to minimize the experimenter's 
utility, any stage other than the full replicate may be chosen giving a 
utility of zero. Thus, any strategy at all that the experimenter uses will 
be a mini -max strategy. 

Two less aggressive strategies for nature are conceivable. The first assumes 
that nature will still choose to have every parameter nonzero but will stop 
the experiment at one of the stages according to prior probabilities known 
by the experimenter, ^hen the only reasonable approach for the experimenter 
is to choose a design which maximizes the utility of the full factorial. 

Clearly this only involves minimizing losses due to block confounding. 


The second of these less aggressive strategies allows that nature will choose 
the parameters to be nonzero according to probabilities known to the experi- 
menter but will choose the stopping stage so as to minimize the experimenter's 
maximum utility. To find the mini -max strategy, the tree form of the game 
can be reduced to the form of a rectangular game and the technique of linear 
programming applied to determine the optimal strategies. Considering the 
size of the tree in many problems it does not seem likely that this approach 
would be computationally feasible. 

Assignment of Utility Functions 


The utility function is a function defined upon the space of all possible 
outcomes of a decision problem. This function describes the absolute 
or relative value to the decision maker of each of these outcomes . Methods 
of specifying the utility function based upon the axioms of utility theory 
are given in Raiffa (ref. A) and DeGroot (ref. 5). These methods involve 
choices of lotteries and would be extremely impractical and cumbersome 
for most problems. Instead, the following five utility functions are 
offered as being both practical and somewhat representative of the decision 
makers preferences in the appropriate instances. 


1. u.(h) 

1 


if estimator unbiased 
if not 


( 13 ) 


This is a utility function which simplifies the expected utility to be the 
probability of getting an unbiased estimate of the parameter. This might 
be a useful utility function in the more basic scientific exploration 
where relative evaluations of the importance of specific parameters are not 
possible . 
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u . ( h ) 

i 




if unbiased 
if unbiased 


( 14 ) 


This simple utility function can be thought of as representing the prior 
probability that an estimator will be unbiasedly estimating a nonzero 
quantity. Such a utility function might be used in response surface or 


optimum seeking experiments. This is 
tributes nothing toward being able to 
levels of the independent variables . 


3 ■ 


u.(h) = 


£ 




if unbiased 
if biased 


true because a zero parameter con- 
change the response by changing the 


( 15 ) 


th 


where n, = the number of treatment combinations at the h w ~ stopping point. 
This utility function could be useful in the situation where the experi- 
mental error is large and the cost of many observations is not much more 
than the cost of a few observations. The rationale for this is that the 
variance of an estimator is proportional to the inverse of the number of 
observations. Thus, one way to. weight the value of an unbiased estimate 
is to weight ^t proportionately to the inverse of the standard deviation, 
that is, 



if unbiased 


if biased 


(16) 


In the opposite situation to (3), suppose the experimental error is negligible 
but the cost of each observation is large so that the cost of the stage is 
a first degree function of the number of treatments. Then this utility 
function would weight the estimators more heavily at the early stages of the 
design and penalize the later stages. 


UjXh) 



if unbiased 


if biased 


( 17 ) 


This is an elementary combination of functions (3) and ( 4 ) and is intended 
for situations where both the costs of the observations are large and the 
experimental error is large. 

Furthermore, a utility function might consist of a combination of (l) or 
(2) along with one of (3), (1), or (5). 


CONCLUDING REMARKS 


The two-level fractional factorial designs represent a class of designs of 
experiments yielding a large number of estimates of first-degree effects 



and interactions for a small amount of experimentation. The main disadvan- 
tage of this class of designs is that the estimates are always estimates 
of aliased combinations of parameters. To make conclusions about single 
parameters it is necessary to have some information about the parameters 
from a source other than the experiment. If such information is available 
before the experiment is performed, it may be incorporated into the design 
of the experiment. 

There are many situations in practice in which an experimenter may have 
varying amounts of information concerning the variables he wishes to 
investigate. This paper has developed some optimal design procedures when 
the prior information is 2 

1. For each parameter the experimenter states his prior probability 
that it is not zero, and 

2. For each parameter the experimenter states what it is worth to 
him to obtain an unbiased estimate of it. 

The information and decisions were formulated as a finite decision problem 
and Bayes and mini -max procedures were developed. 

The main components of the problem were (l) the choice of estimator- 
parameter matching, ( 2 ) choice of physical-design variable matching, and 
( 3 ) evaluations of all possible defining parameter groups and sequences of 
subgroups . 

Modified Bayes and mini -max procedures which are computationally feasible 
were developed and some potential utility functions were presented. 

The procedures presented here (described more fully in ref. 6) should prove 
to be of considerable value in application since the information required 
is of a nature that is easily specified and the computations required 
although tedious are amenable to being programmed for a digital computer. 

In fact, a computer program called NAMER (ref. 7 ) has been written and 
documented which determines the Bayes physical-design variable matching 
and the Bayes parameter -estimator matchings for a specified choice of 
d.p.g.'s at each stopping point of the experiment. Anyone wishing more 
information may write the authors at NASA Lewis Research Center, 21000 
Brookpark Road, Cleveland, Ohio 44135- 
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Figure 1. - Flow chart representing the sequence of choices made by the experimenter and nature. 
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